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ABSTRACT 

The CIPP Evaluation Model was originally developed to 
provide timely information in a systematic way for decision making, 
which is a proactive application of evaluation- This article examines 
whether the CIPP model also serves the retroactive purpose of 
providing information for accountability- Specifically, can the CIPP 
Model adequately assist educators, after the fact, to account for 
their decisions and actions? If so, the CIPP Model is a powerful tool 
for making and implementing decisions and for post hoc accounting for 
those decisions and actions. The remainder of the article describes 
the CIPP Model, presents a conceptualization of accountability, and 
analyzes the potential of the CIPP Model for meeting the information 
requirements of accountability. (Author) 
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THE RELEVANCE OF THE CIPP EVALUATION 
MODEL FOR EDUCATIONAL ACCOUNTABILITY 

Overview 

The CIPP Evaluation Model was originally developed as a means to 
systematically provide timely evaluative Information for use in decision 
making. Use of the CIPP Model thus is intended to facilitate educational 
improvement through a proactive approach to evaluation. 

In this symposium I have been asked to consider whether the CIPP 
Model also affords an adequate means for accountability. This question 
concerns a retroactive , as opposed to a proactive, use of evaluation. 
Specifically, does the CIPP Model provide a useful means by which 
educators, after the fact, can adequately account for their decisions and 

i 

actions? If so, the CIPP Model provides a powerful tool to educators both 
for making and implementing decisions and for post hoc accounting for 
those decisions and actions. 

I believe that the CIPP Model does provide a sound framework for 
both proactive evaluation to serve decision making and retroactive evalua- 
tion to serve accountability. I welcome this opportunity to explain and 
test my reasons for this position. 

The first part of my presentation will attempt to acquaint you with 
the essential features of the CIPP Model. The second will analyze the 
relevance of the CIPP Model for decision making and accountability in 



educational agencies . 
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Sets of handouts have been distributed to assist you in following 
my presentation. Please refer to the first one. 

Introduction to the CIPP Evaluation Model 

This first chart is provided as a convenient list of references to 
the CIPP Model, to which you may wish to refer in the future . 1 It is to 
be noted that the CIPP Model is a comprehensive approach to evaluation. 

It has been developed through individual and group conceptualization 
efforts and derived its major empirical base from the work of The Ohio 
State University Evaluation Center in developing and installing the 
Department of Evaluation and Research in the Columbus, Ohio Public 
Schools . For more complete treatment of the model than can be provided 
here, you are referred to the list of references in your handouts. The 
most comprehensive of these references is the Phi Delta Kappa-sponsored 
book, Educational Evaluation and Decision Making, which is to be 
published next month by the Peacock Publishing Company. 

Evaluation Defined 

Now please refer to the second of your handouts. This chart includes 
the definition of evaluation which is basic to an understanding of the 
CIPP Model. You will note that evaluation is defined as THE PROCESS OF 

^his list of references appears at the back of the paper. 
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DELINEATING, OBTAINING, AND PROVIDING USEFUL INFORMATION FOR 
JUDGING DECISION ALTERNATIVES . There are three important points in 
regard to this definition. First, evaluation is conceived of as a 
systematic, continuing process. Second, the evaluation process 
includes three basic steps: the delineating of questions to be answered 
and information to be obtained, the obtaining of relevant information, and 
the providing of information to decision makers so that they can use it to 
make decisions and thereby to improve ongoing programs. Third, evalua- 
tion is conceived of as a process to serve decision making. Hence, 
proper implementation of the CIPP Model requires understanding of 
educational decision making and procedures for projecting decisions to 
be serviced. 

This definition is further illustrated in your third handout. As shown 
in that chart, activities are evaluated to influence decisions, which 
influence activities, which are in turn evaluated, ad infinitum. The loop 
to the right of the evaluation block in the chart reminds one that the 
evaluation process always includes three steps: delineating the information 
to be collected, obtaining the information, and providing the information 
to decision makers . 

The Framework for the CIPP Model 

Based upon the given definition of evaluation, it is possible to 
derive the basic framework for the CIPP Evaluation Model. That framework 
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is depicted in your founh chart. Two key dimensions have been combined 
to form a matrix as the basis for the CIPP Model. The vertical dimension 

« 

includes the three steps in the evaluation process called delineating, 
obtaining, and providing, while the horizontal dimension includes four 
kinds of evaluation, called context, input, process, and product. The 
acronym CIPP was derived from the first letters of the names of these four 
kinds of evaluation. I have already described the dimension which includes 
delineating, obtaining, and providing as three steps in the evaluation 
process . Now I will comment further about the four kinds of evaluation. 
Then we can consider how the two dimensions interact to form the basic 
framework for the CIPP Model. 

Since evaluation should serve decision making, it is necessary to 
know the decisions to be served . According to the CIPP Model there are 
four kinds of decisions, called planning, structuring, implementing, and 
recycling, which respectively are served by context, input, process, and 
product evaluation. These four evaluation types are portrayed in your 
fifth handout in relation to the four types of decisions. 

Context evaluation provides information about the strengths and 
weaknesses of a total system to assist in planning improvement-oriented 
objectives at each level of the system. Input evaluation provides informa- 
tion about the strengths and weaknesses of alternative strategies which 
might be chosen and structured for the achievement of given objectives. 
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Process evaluation provides information about the strengths and weaknesses 
of a chosen strategy under conditions of actual implementation, so that 
either the strategy or its implementation might be strengthened. Product 
evaluation provides information for determining whether objectives are 
being achieved and whether the change procedure which has been employed 
to achieve them should be continued, modified, or terminated. Basically, 
then, the CIPP Model has been developed to answer four kinds of questions: 
What should we do? How should we do it? Are we doing it correctly? 

and Did it work? 

Given these general descriptions of the four kinds of evaluation, 
we can now discuss each kind further in relation to the three steps in the 
evaluation process. For that purpose, please refer to your sixth handout. 

Context Evaluation 

As noted , the purpose of context evaluation is to systematically 
provide information that can be used by decision makers to make planning 
decisions regarding the establishment of new objectives, modification of 
existing objectives , or confirmation of present objectives . To fulfill this 
purpose a systematic context evaluation program must delineate, obtain, 
and provide appropriate information iri time to make planning decisions . 

De lineating . Delineation of context evaluation should include 
on-file records of the operational specifications and goals of the major 
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programs of the institution, and projections of the "planning" decisions 
that must be made with respect to each of these programs during both 
the immediate and the longer-range future. Anyone calling upon the 
context evaluation section of the institution should be able to obtain 
printed descriptions that include specific objectives and procedures that 
are projected for each of the institution's programs; and they should be 
able to obtain an annotated list of the projected future decisions to be 
made with respect to the objectives for each program in the institution. 

Another delineating activity is systematic contact between the 
contf xt evaluators and decision makers for the purpose of identifying 
emergent problems that might require decisions to change objectives or 
priorities in the institution. 

Obtaining . To aid planning decisions, information must be obtained 
which identifies unmet needs , unused opportunities , and problems.. An 
ongoing program of data collection is needed with respect to the achieve- 
ment of institution objectives at the overall institution level and at the 
level of each of the programs in the institution. This information should 
be categorized and stored systematically, such that decision makers 
could, upon request, receive profiles which explain generally how well 
the institution and its subunits are achieving institutional goals . 

The context evaluation files should contain up-to-date lists of 
unmet needs that should be serviced by the system. These lists should 
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be available to any ad hoc group which is organized to study needs and/or 
problems in the institution. Maintenance of such up-to-date lists should 
prevent redundant data-generating activities . 

Data should also be obtained and filed in retrievable form relative 
to opportunities that might be used to achieve institution objectives. 

Such data would center on, but not be limited to, funding opportunities 
to support institution programs. It would also seem essential to obtain 
and record information about the nature and effectiveness of relevant 

innovative efforts in other institutions . 

It cannot be overemphasized that in collecting context evaluation 
information, the perceptions of the institutional constituencies should be 

m 

surveyed and analyzed • Planners in the institution must be aware of how 
their products, whether from research, development, instruction, or 
leadership activities, are perceived and employed outside the institution. 

Providing . Context evaluation reports should be provided annually 
to all decision bodies in the institution being served. Such reporting 
activities should include both the dissemination of printed material and 
face-to-face oral presentations to particular decision groups to assist in 
interpreting the information relative to particular programs. Such decision 
groups could include boards of education, administrative cabinets, groups 
of principals or individual school principals, project directors, supervisors, 
teachers , students, and parent groups. Annual reports to the faculty of an 
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institution at large might well take the form of profiles which describe 
the performance of an institution as a whole in terms of the institutional 
goals, and specific profiles which describe the performance of each 
institutional program with respect to its objectives . Context evaluators 
should work closely with the institutional programs so that the information 
provided by such profiles could be used to improve institutional programs . 

Input Evaluation 

Next, we turn to input evaluation which has as its purpose to 
identify and assess alternative program strategies for achieving given 
objectives and to provide information to assist in detailing particular 
strategies . To fulfill these purposes, an input evaluation unit must possess 
personnel, resources, and procedures to be used in conducting ad hoc input, 
evaluation studies after a decision which specifies new objectives . Then 
it is necessary to inquire how the chosen objectives can be efficiently 
and effectively achieved. 

Delineating . The delineating step for an input evaluation involves 
the translation of given objectives into criteria and alternative procedural 
strategies . These should be worked out jointly by those who have set the 
objectives and program personnel. The input evaluation team will assess 
alternative strategies, but will not formulate them. A complete record 
should be developed concerning the outputs of the delineating steps. 
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Obtaining . Obtaining is the gathering and analysis of criterion 
information for each of the alternative strategies which was specified 

m 

during the delineating step of input evaluation. 

To obtain such information, reports should be developed for each of 
the identified strategies, which reflect their strengths and weaknesses 
relative to the given objectives . These reports should contain statements 
by both advocates and adversaries of the strategies . Also, they should 
reference relevant research and development literature pertaining to past 
use of the strategies. In some cases where a very expensive program 
might be under consideration, it would also be desirable to obtain pilot 
test information for the competing strategies. Under such conditions 
quasi-experimental designs could be employed . 

Providing . The evaluation unit should report input evaluation 
information to the decision makers in the form of individual reports for 
each of the competing strategies. Further, there should be an analysis 

i 

of the strengths and ease of use of each strategy relative to achievement 
of the given objectives. If a strategy aids achievement of one objective, 
but hinders another, the relative effect of the strategy on the overall 
program should be analyzed . 

Process Evaluation 

Process evaluation is designed to provide information during the 
implementation stages of a project or program,’ which can assist program 
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managers to operate th© program according to its d©sign/ improv© th© 
program design as effects are indicated under operating conditions, and 
to make structuring decisions which could not be made during the prepara- 
tion of the program design. A secondary purpose of process evaluation 
is to provide a complete description of the actual program activities. 

Such a description should be prepared to assist program replication and 
to assist in determining why program objectives were or were not achieved. 

Delineating . The delineating step for a process evaluation involves 
identification of potential procedural barriers, structuring decisions that 

will have to be delayed until the program activities are under way, and 

| 

the major features of the program design for which descriptive information 
should be obtained . This delineating step should be conducted after a 
program design has been developed by those responsible for implementing 
the program. The focus of the delineating activity is the approved program 
design. 

Obtaining . Information to be obtained in process evaluation involves 
a daily monitoring of project activities in accordance with variables 
identified in the delineating step. Techniques which can prove useful for 
monitoring include daily logs, observation, interviews, questionnaires, 
open-end reaction forms, and so forth. A complete file of process data 
should be maintained . 
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Providing . Process dsta should be provided regularly to project 
or program managers. Such information should be provided whenever it is 
needed for preprogrammed decisions or the removal of procedural barriers . 

In some cases the feedback can be daily, as in a special short-term 
training institute. In others it might be weekly or even monthly. At the 
end of a project or program cycle the process evaluator should prepare a 
report which (1) describes the actual procedure that occurred and 
(2) identifies and assesses discrepancies between actual procedure and 
the procedure specified in the original program design. 

Product Evaluation 

The purposes of product evaluation are to relate outcomes to objec- 
tives and to assess the overall worth of a procedure in terms of its effects. 

Delineating . Variables for product assessment should be delineated 
in terms of the objectives which have been selected and in terms of the 
overall problems that a project or program has been designed to solve. 

The product assessment person and the program personnel should define 
criterion variables which relate directly to objectives. 

Obtaining . Product information should be obtained by taking both 
interim and final measures of product criterion variables. To the extent 
possible, such measures should be obtained so that product and context 
data can be compared. In determining the extent to which objectives were 
achieved, one should consider the effect of the product on the overall 
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needs or opportunities which motivated the development of the objectives . 
Major approaches to product evaluation use true experimental design, 
quasi-experimental design, and comparison of products achieved with 
specified standards. 

Providing . Product evaluation reports should be developed and 

communicated both during and after a project or program cycle. Such 

y* 

reports should provide both descriptive and judgmental information about 
project achievements. Achievements should be analyzed in terms of the 
extent to which the intended design was carried out . If satisfactory products 
are not being achieved, it will be important to consider process information 
which would indicate whether or not the designed procedure had been 
implemented as intended. 

This concludes my description of the basic framework for the CIPP 
Evaluation Model. In the next section I will provide my analysis of the 
relevance of the model to accountability. Before moving to that section, 
however, I want to emphasize that what you have just heard is merely a 
brief description of the basic rationale for the CIPP Model. It is by no 
means the complete model. For an in-depth understanding of the model 
I would again refer you to the references which are listed in the first chart 
in your set of handouts . 
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Relevance of the CIPP Evaluation 
MnHpl to Accountability 




Accountability Defined 

In my vocabulary, accountability means the ability to account for 
past actions in terms of the decisi o ns which precipitated the action s , 
the wisdom of those decisions , the ext ent to which they were adequately. 
and efficiently implemented, and the value of their effects . 

Given this definition, a person who is responsible for an action 
program should be able to give defensible answers to sets of questions 
concerning both the ends and means of his program. The answers should 
be defensible in light of present scientific and technological knowledge; 
in terms of some explicit set of moral, social, institutional, and 
individual values; and in terms of appropriate performance data . 

Several questions should be addressed concerning ends. What 
objectives were chosen? What was the wisdom of those choices? How 
adequately did program personnel pursue the chosen objectives? How 

well were the objectives achieved? 

Questions concerning means refer especially to program designs. 
What designs were chosen? Were they chosen for good and sufficient 
reasons ? To what extent were they properly implemented? Of what value 

were their primary, secondary, and tertiary effects? 

Given this conceptualization of accountability, we can now consider 
the question of the relevance of the CIPP Model for meeting accountability 

needs. 
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Implications for Accountability in the CIPP Model j 

Two charts have been prepared for that purpose . Please direct your 
attention first to Chart No. 7 labeled "The Relevance of the CIPP Model 
to Decision Making and Accountability." 

That chart has two main dimensions — context, input, process, and 
product evaluation across the top and different uses of evaluation down 
the side, including both decision making and accountability. 

The main decisions which are serviced by the CIPP Model are 
summarized in the first row of the matrix. Since you are already familiar 
with these, I shall not go over them again. I will instead analyze the 

i 

second row in the chart which refers to accountability. • 

i 

; 

As shown there, context evaluation provides a record of objectives \ 

i 

which were chosen in the past and the bases for their choice. This, I 

think, is the fundamental kind of accountability. When outsiders, 

including the community, representatives of funding agencies, and external 

evaluators, come into. a system and pose basic questions about the value ] 

of objectives being pursued in a system, certainly educational practitioners 

need to be abie to identify their objectives and the rationale for those 

objectives. What are the objectives? Why were they chosen? What 

assumptions do they make? Especially about the needs of children to be 

served? Are those assumptions internally consistent? Are they true? 
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Are they morally, socially, and scientifically valid? Certainly these 
are critical questions, questions that educators should be prepared to 
answer. Context evaluation, I think, provides a basic means to help 
educators in answering these questions . 

Next let us look at accountability with respect to input evaluation. 

As noted, input evaluation should provide a record of chosen strategies 
and designs, as well as the reasons for their choice. Why was a parti- 
cular Title I project design developed? Was it because it provided the 
most promising approach to the achievement of important objectives? Was 
it because some influential funding agent favored the approach? Was 
information available to indicate that it was better than some other 
alternative or set of alternatives? What kind of information was available? 
Did it include evidence of past effectiveness for such a strategy? Were 
cost data available? Was some information available to indicate how 
compatible that particular strategy would be in the particular system in 
which it was to be installed? Educators obviously need to be prepared to 
answer these questions if they are not to be subject to charges of 
irresponsible spending, or merely being too responsive to current fads 
and political pressures. 

As noted for process evaluation, a record of the actual process as it 
occurred would be available through use of the CIPP Model for accountability 
purposes. If a particular procedure was not successful, was it not 
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successful because the project design was never in fact implemented, 
or was it because the design, though implemented correctly, was simply 
inadequate to achieve desired objectives? If we but reflect on the 
controversy that surrounded the early experiments with modern mathematics, 
I think we can understand the importance of process evaluation-type 
information for accountability. You will recall that many persons asked 
whether the "no significant difference" findings comparing modern and 
traditional mathematics curricula were in fact due to modern mathematics 
curricula being no better than traditional curricula , or to the fact that 
teachers actually had never implemented the new modern math curriculum. 
Process evaluation was needed for accountability with respect to those 
experiments . 

Finally, the CIPP Model calls for a record of attainments and of 
decisions about procedures based upon information about the achievements 
of those procedures. If a particular project was continued year after 
year, was it because that project in fact had been successful in achieving 
its objectives or was it merely because someone interested in the 
procedure was still in the system and wanted to perseverate in carrying 
it through? Or might it have been because more Federal money was 
available for that procedure, irrespective of its effectiveness? On the 
other hand, if a procedure were terminated, was it terminated merely 
because of a lack of funds from the outside,; or was it terminated because 
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it in fact had not worked? These obviously are important accountability 
questions. Product evaluation studies provide the kind of information 
needed for this type of accountability. 

Based upon the analysis of Chart 7, the CIPP Model obviously has 
relevance for accountability . For a more complete analysis it is 
necessary to consider whether the CIPP Model meets the data requirements 
that are implied in this paper's conceptualization of accountability. For 
that purpose please refer to Chart 8. 

That chart is a matrix comprised of two dimensions — the four kinds 
of evaluation and the basic data requirements for accountability. Check 
marks in the cells of the matrix indicate the basic data requirements that 
are met by each kind of evaluation . 

As shown, all specified data requirements are met by the CIPP Model. 
Context evaluation identifies objectives that were chosen, the reasons 
that they were chosen, and the goal-related reasons for the choice of 
procedural designs . Input evaluation indicates whether stated objectives 
were the ones that were actively pursued, what particular designs were 
selected, and why they were chosen over other alternatives. Process 
evaluation confirms further whether stated objectives were actually pursued 
and whether procedural specifications were actually implemented. Product 
evaluation reveals whether objectives were achieved and what main and 
side effects resulted from the implementation of a project. 
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Use of the CIPP Model to Meet Accountability Needs 

In Chart 8 there is further indication that the CIPP Model provides 
a powerful framework for meeting accountability needs. However, two 
further points need to be made concerning how the CIPP Model must be 
used if it is to provide accountability in education. 

First, no matter how well internal evaluation is performed, no matter 
how completely the CIPP Evaluation Model is implemented, there is still 
the need for outside, independent audits and checks on the system. 
Outsiders should be brought in periodically and invited to ask hard 
questions, to make judgments, and in general to provide an outside, 
external, summative kind of evaluation with respect to a system's goals, 
designs, procedures, and results. Such an outside evaluation can be 
performed much more thoroughly if an internal evaluation group is 
performing the same function internally and thereby providing some of the 
basic data for the external evaluation. Obviously there will be times 
when the internal evaluation group will not be credible with respect 
to certain audiences for its evaluation reports. Those are important 
occasions when an outside opinion is absolutely necessary. 

In relation to this point, it is further to be noted that there is much 
to argue for a cybernetic relationship between an internal evaluation unit 
and all of the decision-making levels in a system.* It is highly important 



*1 am indebted for thi/3 important point to Dr. Patrick Tool. 
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that evaluation service decisions at all levels of the system and that 
information not be screened and filtered through one particular bureaucratic 
level. This will be a hard change to make in many institutions, but one 
which I think can lead to a substantial freeing of a system and to more 
responsibly autonomous performance on the part of persons lower down 
in the system but obviously much nearer to the educational action . 

Conclusion 

In conclusion, I think that the CIPP Evaluation Model promises a sound 
accountability system, both for ongoing normal efforts of a system and for 
change efforts in that system. In that vein I recommend the CIPP Model 
to you. Not only does it provide post hoc information for accounting for 
past decisions and past actions, but also in a formative sense it provides 
information proactively to decision making so that decision makers can be 
more rational in their decisions in the first place. The system which 
provides such a powerful combination would, it seems to me, be a great 
improvement over social accounting and standardized test information 
systems which are typically found in schools, colleges of education, 
government education agencies, and other education agencies. 

Summary 

To summarize, I have attempted to address the issue of the relevancy 
of the CIPP Evaluation Model to concerns for accountability. First I 
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defined the CIPP Model as a function of two basic dimensions, including 
three steps in the evaluation process called delineating, obtaining, and 
providing, and four kinds of evaluation called context, input, process, 
and product. Then I described the particular accountability needs that 
are served by each of the four kinds of evaluation. I also urged that both 
external and internal evaluations are needed to service both decision- 
making and accountability needs, and that internal evaluation should have 
a cybernetic relationship to decision-making levels. My concluding 
opinion was that the CIPP Evaluation Model provides a sound evaluative 
framework to service both decision-making and accountability needs. 



Thank you for your attention . 
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Definition: 

• 


EVALUATION IS THE (1. PROCESS) OF (2. 
DELINEATING), (3. OBTAINING), AND (4. 
PROVIDING) (5. USEFUL) (6. INFOR- 
MATION) FOR (7. JUDGING) (8. DECI- 
SION ALTERNATIVES). 


Terms: 


1. Process. A particular, continuing, and 

cyclical activity subsuming many methods . 

and involving a number of steps or oper- 
ations. 

i 

2. Delineating. Focusing information require- 
ments to be served by evaluation through 
such steps as specifying, defining, and 
expl i eating. 

3 . Obtaining. Making available through 
such processes as collecting, organiz- 
ing, analyzing, and reporting, and 
through' such formal means as statistics 
and measurement. 

4. Providing. Fitting together into systems 
or subsystems that best serve the needs 
or purposes of the evaluation. 

5 . Useful. Appropriate to predetermined 
criteria evolved through the inter- 
action of the evaluator and the cl ient. 

6. Information. Descriptive or interpretive 
data about entities (tangible or intangi- 
ble) and their relationships. 

7 . Judging. Assigning weights in accordance 
with a specified value framework, criteria 
derived therefrom, and information which 

relates criteria to each entity being . 

judged. 

8. Decision Alternatives. A set of optional 
responses to a specified decision question. 

Chart 2-. EVALUATION DEFINED 
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Chart 3: THE RELATIONSHIP OF EVALUATION TO DECISION MAKING 
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Chart 5! EVALUATION AND DECISION TYPES 
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Chart 7 : THE RELEVANCE OF THE CIPP MODEL 

TO DECISION MAKING AND ACCOUNTABILITY 



